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Development  and  Utility  of  the  Front  Line  Manager’s 
Quick  Reference  Guide 


introduction 

Air  traffic  control  (ATC)  Front  Line  Managers  (FLMs) 
are  instrumental  in  meeting  Federal  Aviation  Administration 
(FAA)  Flight  Plan  objectives  for  mitigating  Operational 
Incidents  (OIs)  and  Runway  Incursions  (RIs)  by  providing 
leadership  to  ensure  that  air  traffic  controllers’  performance 
meets  FAA  goals  for  improving  the  safety  of  the  National 
Airspace  System  (NAS).  FLMs  also  influence  the  preven¬ 
tion  of  errors  by  enabling  exemplary  controller  performance 
(Greene,  2010). 

Information  to  assist  air  traffic  control  FLMs  in  the 
performance  of  their  technical  duties  may  not  be  readily 
available,  and  formal  training  may  be  delayed  due  to  a 
number  of  factors  such  as  access  to  organizational  training 
or  operational  mentorship.  Providing  FLMs  with  informa¬ 
tion  that  helps  them  supervise  controllers  may  impact  the 
overall  safety  of  air  traffic  operations.  The  Quick  Reference 
Guide  (QRG)  was  designed  as  a  resource  to  help  ATC  FLMs 
supervise  their  employees  more  effectively  by  promoting 
safety,  enhancing  communication,  and  encouraging  respect 
in  a  way  that  enhances  controller  performance.  The  QRG 
is  available  to  the  FLMs  online  at: 
https://employees.faa.gov/org/linebusiness/ato/operations/ 
media/Front_Line_Manager_Quick_Reference_Guide.pdf 

It  is  also  available  in  hard  copy  in  FAA  field  offices. 

Air  Traffic  Control  Front  Line  Managers 

ATC  FLMs  provide  first-line  supervision  to  teams  of 
developmental  controllers  and  certified  professional  con¬ 
trollers  (CPCs).  FLMs  are  responsible  for  planning  and 
assigning  work  priorities,  evaluating  and  recommending 
performance  ratings,  and  listening  to  and  resolving  com¬ 
plaints  of  controllers  under  their  supervision.  FLMs  are  also 
responsible  for  identifying  and  arranging  for  the  training 
needs  of  their  staff  as  well  as  recommending  goals  and 
objectives  and  the  means  to  track  these  accomplishments. 
They  also  participate  in  the  labor-management  process  and 
foster  an  equitable  working  environment.  FLMs  must  be 
excellent  communicators  and  be  able  to  adjust  quickly  to 
a  changing  work  environment.  FLMs  use  a  combination 
of  these  skills  in  supervising  controllers.  These  skills  are 
also  intended  to  mitigate  OIs  and  RIs  (International  Civil 
Aviation  Organization,  2005). 


The  Air  Traffic  Organization  (ATO)  supported  a  survey 
to  evaluate  the  QRG’s  efficacy.  This  report  describes  the 
development  of  the  Air  Traffic  Control  Front  Line  Manag¬ 
ers  Quick  Reference  Guide  and  summarizes  the  findings 
of  the  survey  designed  by  the  Civil  Aerospace  Medical 
Institute  (CAMI)  to  examine  the  operational  functionality 
of  the  QRG. 

Development  of  the  Front  Line  Manager  Quick 
Reference  Guide 

Booz  Allen  Hamilton  (BAH)  conducted  studies  spon¬ 
sored  by  the  FAA’s  Human  Factors  Division  and  Engineer¬ 
ing  Division  (ANG-C1)  and  the  ATO  Offices  of  Safety 
(ATO-S),  En  Route  and  Oceanic  (ATO-E),  and  Terminal 
Support  (ATO-T)  services  to  identify  best  practices  of  FLMs 
for  promoting  safety  and,  therefore,  increase  the  likelihood 
of  reducing  the  occurrence  of  OIs  and  RIs  throughout  the 
National  Airspace  System. 

Background  research  for  the  QRG  was  developed  in 
three  phases,  using  different  methodology  for  each  group 
of  FLMs:  en  route,  tower,  and  terminal  radar  approach 
control  (TRACON).  The  first  FLMs  to  be  assessed  were 
en  route,  followed  by  tower,  and  TRACON. 

En  route  QRG  development.  The  en  route  QRG 
background  research  was  conducted  and  reported  in  2006 
(Duley,  Capriglione,  Kalita,  Guenther,  &Nagy,  2006a).  This 
research  focused  on  general  supervisory  best  practices  used 
by  exemplary  FLMs  in  response  to  a  variety  of  identified 
problems  encountered  in  the  areas  of  selection  and  assign¬ 
ment,  skills  and  knowledge,  motivation  and  incentives, 
and  operational  environment.  BAH  used  the  Optimizing 
Human  Performance  (OHP)™  Front  End  Analysis  (FEA) 
methodology  as  a  human  performance  model  to  identify 
performance  with  the  goal  of  improvement  (Saba  Software, 
2001).  The  components  of  optimal  performance  were 
identified  as  skills  and  knowledge,  motivation  and  incen¬ 
tive,  and  operational  environment.  The  FEA  approach  also 
allowed  for  linkage  to  other  systems  such  as  rewards,  learn¬ 
ing  management,  and  assignment  of  FLMs.  BAH  provided 
certified  expertise  with  this  methodology. 

Within  the  FEA  framework,  BAH  reviewed  literature 
related  to  OIs  within  the  en  route  environment  and  col¬ 
laborated  with  ATO-E  to  identify  exemplary  FLMs  and 
controllers  to  serve  as  subject  matter  experts  (SMEs)  to 
provide  expertise.  Next,  BAH  conducted  working  group 
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Table  1 .  QRG  Problem  Format  Example 


Problem  Statement: 

Consistent  performance 
management  practices  are 
not  always  utilized 

FLM  Challenges: 

1 .  Managing  performance  and  conduct  of  controllers  who  are  not  under 
the  watch  supervisor’s  administrative  responsibility 

2.  Knowing  the  performance  history  or  background  of  another 
supervisor’s  controllers  to  effectively  manage  their  performance 

3.  Inconsistently  documenting  both  positive  and  negative  feedback 

4.  Knowing  what  constitutes  correct  performance 

5.  Relaying  performance  concerns  and  praises  to  the  supervisor  of  record 

Mitigation  Topic: 

Mitigation  Topic: 

Mitigation  Topic: 

Fostering  a  Positive 
Work  Culture 

Accountability 

Communication 

Mitigation  Topic: 

Mitigation  Topic: 

Active  Listening 

Sharing  Information 

meetings  to  establish  project  expectations  and  develop 
lists  ofFLM  target  job  duties  and  responsibilities.  Manag¬ 
ers  designated  as  SMEs  were  then  interviewed  to  validate 
project  expectations  with  levels  ofFLM  target  duties  and 
responsibilities. 

Prior  to  site  visits,  BAF1  reviewed  relevant  FAA  docu¬ 
ments,  OI  reports,  and  other  related  reports.  Site  visits 
were  conducted  at  six  Air  Route  Traffic  Control  Centers 
(ARTCCs)  for  structured  FEA  interviews  with  FLMs  and 
reviews  of  site  documentation  relevant  to  safety  and  best 
practices.  BAF1  focused  on  analyzing  factors  identified  as 
being  both  barriers  to  and  facilitators  of  accomplished 
performance.  Consistent  with  the  OF1P  process,  the  data 
collected  were  organized  around  the  identified  components 
of  optimal  performance:  selection  and  assignment,  skills 
and  knowledge,  motivation  and  incentive,  and  operational 
environment.  Since  both  barriers  to  and  facilitators  of  best 
practices  were  reported,  patterns  in  the  data  permitted  fur¬ 
ther  categorization  into  problem  statements  that  focused  on 
factors  facilitating  best  practices  and  factors  hindering  best 
practices.  The  SMEs  agreed  that  these  workplace  problem 
areas  were  common  to  the  ATC  work  environment.  With 
problem  areas  reported,  viable  workplace  interventions  or 
solutions  were  also  identified  and  documented  for  each 
problem  statement. 


Identified  ATC  FLM  problem  areas  with  viable  solutions 
became  the  format  for  the  development  of  the  En  Route 
QRG  (Duley,  Capriglione,  Kalita,  Guenther,  &  Nagy, 
2006b).  Table  1  illustrates  this  problem  statement  format. 
The  QRG  was  developed  to  provide  precise  management 
focus  to  the  ATC  working  environment  with  the  intent  of 
preventing  and  mitigating  both  OIs  and  RIs. 

Tower  QRG  development.  The  2008  Tower  FLM  study 
used  the  Threat  and  Error  Management  methodology  to 
identify  best  practices  to  prevent  errors  and  their  contribut¬ 
ing  factors  (Capriglione,  Williams,  Daniels,  &  Phillips, 
2008) .  The  model  described  the  relationship  between  human 
performance  and  error  in  a  conceptual  framework.  Prior 
to  site  visits,  BALI  reviewed  OI  and  RI  data  and  literature 
related  to  OIs  and  the  manager’s  role.  Through  site  visits 
to  1 6  air  traffic  control  towers  and  through  interviews  with 
30  FLMs,  factors  influencing  supervisor  effectiveness  in  the 
tower  environment  were  defined.  Relevant  coordination,  com¬ 
munication,  and  memory  best  practices  were  documented. 

Specifically,  coordination  focused  on  controller-to- 
controller  interactions  and  transfer  of  information. 
Communication  focused  on  controller-to-pilot  exchange 
of  information.  Memory  focused  on  controller  recall  of 
procedures  and  actions,  as  well  as  situational  awareness. 
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Table  2.  QRG  Content  Example 


Error  Identification  and 
Mitigation: 

FLM  Coordination  Error: 

1 .  Problem  Detection 

2.  Best  Practices 

Coordination  Error 

Mitigation 

Team  Building 

Mitigation 

Conflict  Resolution 

Mitigation 

Observation 

Mitigation 

Active  listening 

In  addition  to  identification  of  controller  best  practices  in 
these  areas,  this  study  identified  a  three-step  process  used 
for  applying  these  practices:  1)  problem  detection,  2)  causal 
determination,  and  3)  problem  mitigation  or  elimination 
response.  Table  2  provides  a  QRG  content  example  for 
coordination  errors. 

The  tower  study  identified  FLM  error  management  best 
practices  in  the  tower  operational  environment;  however, 
much  of  the  information  was  relevant  to  all  FLMs.  While  the 
information  provided  permitted  an  expanded  best  practices 
format,  it  also  provided  the  identification  ofspecific  problem 
mitigation  topics  to  be  included  in  the  QRG,  along  with 
identification  and  mitigation  techniques. 

To  incorporate  this  information  in  the  Tower  version 
of  the  QRG,  BAF1  expanded  the  ATC  supervisory  best 
practices  identified  by  the  en  route  study  and  added  some 
best  practices  for  tower  FLMs  identified  in  the  tower  study. 
Specifically,  the  Problem  Statement  section  was  expanded 
to  include  tower-relevant  situations  and  mitigating  factors, 
as  well  as  expand  the  set  of  best  practices  applicable  to  all 
FLMs. 

TRACON  QRG  development.  The  third  study,  which 
focused  on  TRACON  FLMs,  was  completed  by  BAF1  in 
2010  (Floenicke,  Rudolph,  &  Hill,  2010).  The  BAF1  team 
conducted  a  Gap  Analysis  on  information  collected  from  the 
en  route  and  tower  studies  to  identify  factors  common  to 
TRACON.  After  identifying  common  factors  of  situational 
awareness,  training,  compression/complexity,  memory, 
position  relief,  communication,  performance  manage¬ 
ment,  staffing,  equipment,  and  motivation  and  incentives, 
these  common  factors  were  reviewed  for  the  TRACON 
environment  as  related  to  error  prevention.  In  addition, 
the  proportion  of  developmental  controllers  supervised 
and  weather  were  added  as  factors  to  be  considered  in  er¬ 
ror  prevention  and  mitigation.  The  TRACON  study  also 
identified  leadership  skills  used  by  effective  FLMs,  which 
promoted  operational  management  and  administrative 
effectiveness.  This  resulted  in  the  addition  of  a  Leadership 
section,  which  followed  the  Problem  Statement  section  in 
the  QRG. 


QRG  consolidation.  BAF1  research  consolidated  the 
content  information  provided  for  all  three  services  (en  route, 
tower,  and  TRACON).  The  TRACON  information  was 
incorporated  into  the  existing  QRG  draft  from  the  en  route 
and  tower  studies.  The  draft  document  was  made  available 
for  review.  The  FAA’s  Air  Traffic  Supervisor’s  Committee 
(SUPCOM)  provided  SME  review  of  the  document.  The 
original  SME  research  coordinators  were  retained  for  con¬ 
tinuity.  These  reviewers  worked  with  ATO-S,  ANG-C1, 
and  CAMI  to  review  the  BAF1  draft  QRG  document.  The 
document  was  revised  to  remove  some  location-specific 
management  situations,  making  the  overall  document  ap¬ 
plicable  to  managerial  conditions  across  all  facility  types. 
Language  was  carefully  reviewed  and  adjusted  to  reflect 
current  AT  professional  use.  In  addition,  sections  were 
included  for  Crew  Resource  Management,  Safety  Culture, 
AirTraffic  Safety  Action  Program,  and  Additional  References 
and  Resources  as  supporting  documentation. 

To  summarize,  more  than  60  En  Route,  Tower,  and 
TRACON  FLMs  and  their  supervisors  were  interviewed 
from  30  facilities  for  the  entire  project.  Research  and  field 
interviews  focused  on  the  circumstances  that  contribute 
to  errors  and  the  FLM  best  practices  used  to  mitigate  or 
prevent  them.  This  made  the  QRG  particularly  relevant 
to  the  FLM  population  as  a  management  tool.  According 
to  the  Executive  Summary  of  the  2006  Supervisory  Best 
Practices  for  Operational  Incident  Prevention,  “The  guide 
serves  as  a  job-aid  that  addresses  the  gap  between  existing 
Operational  Supervisor  training  resources  that  deal  with 
administrative  functions  but  overlook  the  people  skills 
that  are  critical  for  avoiding  operational  incidents”  (Duley 
et  ah,  2006a,  p.  vi). 

Although  the  QRG  was  developed  as  a  standalone  refer¬ 
ence  tool,  it  was  also  incorporated  into  existing  training  and 
could  be  used  as  a  basis  for  the  development  of  new  training 
specific  to  air  traffic  FLMs.  This  provided  for  identification 
and  applicability  of  management  skills  specific  to  those 
needed  by  air  traffic  FLMs,  with  a  focus  on  the  reduction 
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Table  3.  QRG  Organization 


AT  FLM  Quick 
Reference  Guide 

FLM  Tabbed  Topics: 

1 .  Problem  Statements 

2.  Leadership 

3.  Performance  Management 

4.  Communications 

5.  Mitigating  Coordination  Error 

6.  Mitigating  Communication  Error 

7.  Mitigating  Memory  Error 

8.  Mitigating  Complexity  Error 

FLM  Tabbed  Supporting  Information  Topics: 

9.  Safety  Culture 

10.  Air  Traffic  Safety  Action 

Program  (ATSAP) 

11.  Crew  Resource  Management  (CRM) 

12.  Additional  References  and  Resources 

and  mitigation  of  errors.  Table  3  provides  an  illustration 
of  the  QRG  content  topics  and  supporting  information. 

The  FLM  QRG  was  delivered  to  managers  at  air  traffic 
control  facilities  in  hard  copy  and  to  FLMs  electronically 
in  December  2010.  In  order  to  assess  the  impact,  ATO 
supported  a  survey  to  evaluate  the  efficacy  of  the  QRG. 
This  survey  was  designed  to  evaluate  the  utility  of  the 
FLM-QRG  and  determine  if  additions  would  be  valuable 
to  ATC  FLMs  and  could  be  considered  for  updates  to  the 
QRG  at  a  later  date.  The  survey  was  opened  electronically 
in  June  of  20 1 1  and  remained  open  through  July  of  20 1 1 . 

METHOD 

Survey 

To  assess  the  utility  of  the  QRG  in  the  field,  a  survey 
consisting  of  22  questions  was  distributed  to  all  ATC  FLMs. 
The  survey  was  designed  to  assess  FLM  perceptions  of  the 
QRG,  including  content  organization,  usefulness,  and  ap¬ 
propriateness.  The  survey  also  solicited  feedback  regarding 
potential  content  errors,  problems  with  QRG  utility,  ad¬ 
ditional  content  areas  that  should  be  addressed,  and  general 
suggestions  for  improvement.  In  addition  to  their  perception 
of  the  QRG,  the  survey  collected  demographic  information 
about  the  participants,  including  current  supervisory  status 
(e.g. ,  current  FLM) ,  air  traffic  option  (en  route  or  terminal) , 
terminal  facility  type  (TRACON,  tower,  tower/TRACON), 
facility  level,  and  certified  professional  controller  (CPC), 
controller  in  charge  (CIC),  and  FLM  tenure. 

The  survey  and  its  administration  methodology  were 
submitted  to  and  approved  by  the  CAMI  Independent 
Review  Board  (IRB).  Survey  participant  confidentiality 
was  assured. 


Survey  Participants 

A  list  of  FAA  air  traffic  supervisory  employees  was 
provided  by  the  FAA’s  Information  Technology,  Informa¬ 
tion  Delivery  and  Collaborative  Services  office.  The  list 
(N=3,151)  included  all  supervisory  employees  within  the 
air  traffic  control  specialist  job  series  who  were  assigned  to 
a  facility  responsible  for  the  separation  of  air  traffic.  Super¬ 
visory  personnel  at  the  FAA’s  Washington  Fleadquarters, 
Technical  Center,  and  Aeronautical  Center  were  removed 
from  the  list,  reducing  the  pool  to  2,956  persons.  The  list 
was  further  refined  to  exclude  area  managers,  operations 
managers,  higher-level  senior  managers,  and  nonsupervi- 
sory  managers,  resulting  in  a  sample  of  2,130  FLMs.  Of 
those,  1 1  survey  invitations  were  undeliverable  and  1  FLM 
was  removed  upon  the  supervisor’s  request,  leaving  a  final 
adjusted  sample  of  2, 1 1 8  FLMs. 

Invitations  to  complete  the  survey  were  distributed  to 
FLMs’  internal  FAA  electronic  mail  addresses.  Completed 
surveys  were  voluntarily  submitted  online  by  849  partici¬ 
pants,  resulting  in  a  40.1%  overall  response  rate.  Among 
responders,  58  indicated  that  they  were  not  FLMs  and 
were  excluded  from  further  reporting.  The  remaining  791 
respondents  were  included  in  the  final  reporting  and  analysis. 

Survey  Administration 

The  survey  was  designed  and  distributed  using  Snap 
Survey®  Version  10.21  software.  The  software  enabled  dis¬ 
semination  via  electronic  mail  invitations  with  embedded 
hyperlinks.  An  electronic  mail  message  announcing  the 
survey  and  encouraging  participation  was  distributed  to 
participants’  FAA  email  addresses  on  June  1 ,  2011.  The 
announcement  included  an  embedded  hyperlink  to  view 
and  download  the  FLM  QRG. 
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Invitations  to  complete  the  survey  were  sent  via 
electronic  mail  on  June  15,  201 1.  To  control  access  to 
the  survey  and  protect  respondent  anonymity,  a  unique 
hyperlink  was  embedded  in  each  invitation  and  all  subse¬ 
quent  electronic  mail  reminders.  The  hyperlinks  enabled 
the  survey  software  to  track  survey  completions,  generate 
a  unique  identification  number  in  a  database  to  track 
submitted  surveys,  and  automatically  generate  electronic 
mail  reminders  to  participants  who  had  not  submitted  a 
completed  survey.  Participants  without  completed  surveys 
were  emailed  up  to  four  reminders  at  14-day  intervals. 
The  reminders  encouraged  participation  and  included 
an  embedded  hyperlink  to  access  the  survey.  The  survey 
closed  on  August  1,2011.  Online  survey  responses  were 
downloaded. 

RESULTS 

Most  participants  (564;  71.4%)  indicated  that  they 
had  reviewed  the  QRG.  Of  those  who  had  reviewed  the 
QRG,  the  majority  (72.5%)  reviewed  an  electronic  copy. 
Overall  FLM  perceptions  of  the  QRG  were  positive. 
Most  FLMs  responded  that  the  information  contained 
in  the  QRG  was  somewhat  or  very  appropriate  to  their 
job  (82.8%),  and  79.2%  felt  that  it  was  well  organized. 
Over  half  of  all  FLMs  indicated  that  they  would  ‘prob¬ 
ably  or  ‘definitely’  use  the  QRG  as  a  reference  tool  in  the 
future  (60.8%).  When  split  by  FLM  tenure  (6  or  fewer 
years,  7  to  10  years,  more  than  10  years),  lower  tenure 
(60.6%)  and  higher  tenure  (70.7%)  FLMS  were  more 
inclined  to  use  the  QRG  as  a  reference  tool  for  their  jobs 
in  the  future.  Overall,  85.6%  of  FLMs  reported  that  the 
QRG  was  somewhat  to  very  useful.  Item  level  results  of 
the  survey  are  presented  in  Appendix  A  of  this  report. 

FLMs  were  asked  if  they  had  suggestions  for  improv¬ 
ing  the  QRG.  Sixty-seven  (12%)  indicated  ‘yes’  and 
subsequently  provided  at  least  one  suggestion  for  im¬ 
provement.  Their  verbatim  responses  were  content  coded 
using  21  unique  topics  codes  organized  into  three  major 
areas  related  to  the  QRG  —  Content,  Implementation, 
and  Technology  —  with  a  fourth  content  area  for  com¬ 
ments  not  directly  related  to  the  QRG.  Each  topic  code 
was  assigned  only  once  to  each  comment  item  (items 
10-13)  for  each  respondent,  with  the  number  of  codes 
assigned  varying  from  one  to  five  codes.  A  total  of  148 
codes  were  assigned  to  the  67  survey  respondents  who 
provided  comments  and  suggestions. 

The  majority  of  comments  were  coded  into  the  QRG 
Content  area  (106  codes) .  Within  the  QRG  Content  area, 
most  of  the  comments  indicated  that  the  QRG  was  too 
long  or  contained  too  much  information,  while  some 
provided  suggestions  for  additional  content  areas  such 
as  Lluman  Resources  Content  and  Labor  Management 


Relations  Sections.  Of  the  12%  of  FLMs  who  made 
suggestions  for  improvements,  none  noted  any  content 
errors  in  the  existing  QRG. 

The  QRG  Implementation  coding  content  area  (25 
codes)  contained  comments  focused  primarily  on  pro¬ 
viding  the  QRG  in  a  condensed  pocket  version  or  quick 
guide,  providing  hard  copies  or  electronic  copies  of  the 
guide,  and  comments  about  inadequate  time  or  staffing 
to  use  the  QRG  and  the  need  to  promote  the  QRG  and 
train  FLMs  to  use  it. 

The  Technology  content  area  comments  recommended 
simplifying  the  electronic  version  using  highlighted 
sections,  including  electronic  links  or  bookmarks,  and 
enhancing  or  facilitating  download  and  print  options. 

CONCLUSION 

Survey  data  indicate  that  FLMs  perceive  that  the  QRG 
is  well  organized  and  contains  appropriate  information  to 
be  a  useful  reference  tool.  Both  less  and  more  experienced 
FLMs  find  the  QRG  to  be  helpful,  as  indicated  by  reported 
intended  use,  with  the  more  experienced  FLMs  indicating 
higher  intended  use.  There  were  also  recommendations  for 
improving  the  utility  of  the  QRG  by  making  it  download 
more  quickly  and  providing  enhanced  navigation  tools. 
Recommendations  for  imbedded  links  to  other  relevant 
FAA  documents  were  also  suggested. 

The  QRG  was  first  envisioned  as  a  best  practices  guide 
to  aid  both  new  and  experienced  FLMs  in  dealing  with 
their  job  responsibilities.  Overall,  it  has  served  as  attaining 
support  when  other  training  opportunities  may  not  be 
readily  available.  The  QRG  is  available  both  in  hard  copy 
and  online  and  can  be  kept  in  each  office  where  FLMs  are 
responsible  for  aiding  controllers  in  their  duties.  Organi¬ 
zational  support  has  been  documented  for  the  QRG  in 
FLM  training  and  through  SUPCOM  Workshops,  and 
it  has  been  frequently  accessed  through  the  website,  with 
over  1 ,000  downloads  in  20 1 1 .  Through  incorporation 
into  training  programs  and  website  access,  it  has  become 
an  integral  part  of  the  FLM  toolkit  to  advise  controllers 
on  elements  of  air  traffic  control. 

Individual  areas  of  the  QRG  may  be  considered  for 
updates  and  there  are  several  additional  areas  recom¬ 
mended  for  inclusion  that  would  enhance  utility.  Fatigue 
risk  management  is  one  of  these  areas  recommended  for 
inclusion,  as  well  as  sections  on  human  resources  and  labor 
management  relations.  The  continued  maintenance  and 
utility  of  this  document  will  depend  on  a  commitment 
by  management  and  the  organizational  sponsorship  of 
the  QRG. 
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APPENDIX  A 

Front  Line  Manager  Quick  Reference  Guide  Survey,  2011  Overall  Report 


Federal  Aviation 
Administration 


Front  Line  Manager  Quick  Reference  Guide  Survey 
2011  Overall  Report 


Federal  Aviation  Administration 
Civil  Aerospace  Medical  Institute 
Aerospace  Human  Factors  Research  Division 
Oklahoma  City,  Oklahoma 

And 

Xyant  Technology,  Incorporated 
Oklahoma  City,  Oklahoma 


At 


Explanation  of  Report  Content 


The  Federal  Aviation  Administration’s  (FAA)  Civil  Aerospace  Medical  Institute  distributed  a  survey  to  Air  Traffic  Front 
Line  Managers  (FLMs)  during  June  and  July  2011.  The  survey  was  designed  to  gather  information  about  job 
requirements,  and  examine  managers'  perceptions  of  the  Air  T raffic  FLM  Quick  Reference  Guide  (QRG). 

The  results  of  this  survey  will  be  used  to  assist  the  FAA  and  the  Technical  Operations  Supervisor's  Committee 
(SUPCOM)  in  the  development  of  FLM  training  and  reference  materials.  This  report  provides  the  results  of  the  survey. 


Example  of  Report  Format 


Response  Distribution  (%) 


5.  How  well  organized  is  the  QRG? 


2011 


n  m  sd 

564  3.94  0.99 


3.9 


5.9 


Very  Somewhat 

disorganized  disorganized 


11.0 


Neither 


50.5 


■ 


28.7 


Somewhat  Very  organized 
organized 


Descriptive  Statistics 

Number  of  Respondents  (n).  The  number  of  people  that  provided  a  usable  (i.e. ,  valid)  response  for  an  item. 

Mean  (m).  The  mean  is  the  arithmetic  average,  or  the  sum  of  all  scores  for  an  item  divided  by  the  number  of  people 
who  answered  that  item.  Means  are  reported  for  items  answered  on  interval  scales  (e.g.,  Usefulness).  Each  response 
option  in  the  scale  is  assigned  a  number  from  1  (low)  to  5  (high).  For  example,  on  the  Usefulness  scale,  the  first 
response  option  (Not  useful)  would  be  assigned  a  score  of  1,  and  the  last  response  option  (Very  useful)  would  be 
assigned  a  score  of  5. 

Standard  Deviation  (sd).  The  standard  deviation  is  a  measure  of  dispersion,  or  spread  of  scores  around  the  mean. 
Smaller  standard  deviation  values  indicate  higher  levels  of  agreement  among  respondents. 

Minimum  (Min).  The  lowest,  or  minimum,  score  provided  for  an  item. 

Maximum  (Max).  The  highest,  or  maximum,  score  provided  for  an  item. 

Median  (Med).  The  exact  middle  data  point  in  a  set  of  rank-ordered  scores.  It  is  less  affected  by  extreme  scores  in 
comparison  to  the  mean,  and  thus,  is  relied  upon  when  extreme  scores  are  present  in  a  data  set  (e.g.,  year  became  a 
CPC  controller,  item  17). 

Response  Distributions  (%).  Distributions  can  show  where  perceptions  are  negative  or  positive  by  looking  at  the 
percentage  of  the  respondents  choosing  low  (1  and  2)  or  high  (4  and  5)  response  options.  Items  are  written  so  that  a 
response  of  4  or  5  is  positive. 


A 2 


2011  Front  Line  Manager  Quick  Reference  Guide  Survey 

This  survey  was  designed  to  gather  information  about  job  requirements,  and  to  examine  Front  Line  Managers'  (FLM) 
perceptions  of  the  Air  Traffic  FLM  Quick  Reference  Guide  (QRG).  Therefore,  only  those  respondents  who  indicated  that 
they  were  currently  serving  in  a  supervisory  capacity  at  an  air  traffic  facility  were  included  in  the  final  reporting  for  items  2 
through  22. 

Some  items  were  used  to  branch  or  route  the  respondent  to  different  sections  of  the  survey.  These  items  were  required 
and  are  denoted  following  the  item  text. 


Front  Line  Manager 


Response  Distribution  (%) 


1.  Are  you  currently  an  active  Front  Line  Manager? 
(Required) 


n 

849 


93.2 


Yes 


(not  included  in  items  2-22) 


The  remainder  of  this  report  (items  2  through  22)  includes  only  those  respondents  who  indicated  that  they  were  an  active 
Front  Line  Manager  ('Yes'  on  item  1 ). 


2.  In  prior  years,  do  you  recall  participating  in  a  study 
of  Front  Line  Managers  to  identify  best  practices  for 
promoting  safety  and  preventing  safety  incidents? 

n 

790 


Yes  No 


Quick  Reference  Guide  (QRG) 


3.  Have  you  reviewed  the  Front  Line  Manager  Quick 
Reference  Guide  (QRG)?  (Required) 

n 

791 


71.4 


Yes  No 

(skip  to  item  14) 


Items  4  through  13  include  only  those  respondents  who  indicated  that  they  had  reviewed  the  FLM  QRG  ('Yes'  on  item  3). 


4.  Did  you  review  the  QRG  as: 

n 

564 


53.0 


Hard  copy  Electronic  copy  Both 
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Quick  Reference  Guide  (QRG) 


Response  Distribution  (%) 


5.  How  well  organized  is  the  QRG? 

n  m  sd 

564  3.94  0.99 


6.  How  useful  is  the  QRG? 

n  m  sd 

563  3.37  1.13 


7.  How  appropriate  was  the  information  contained  in 
the  ORG  to  your  job? 

n  m  sd 

554  4.02  0.93 


8.  Do  you  think  you  will  use  the  QRG  as  a  reference 
tool  for  your  job  in  the  future? 

n  m  sd 

554  3.61  1.00 


50.5 


Very  Somewhat  Neither  Somewhat  Very 
disorganized  disorganized  organized  organized 


45.1 


Not  useful  Somewhat  Neither  Moderately  Very  useful 
useful  useful 


52.7 


Very  inappropriate  Neither  Very  appropriate 


42.4 


Definitely  Probably  Unsure  Probably  Definitely 
not  not  yes  yes 


9.  Do  you  have  suggestions  for  improving  the  QRG? 
(Required) 


n 

556 


12.1 

Yes 


88.0 


No 

(skip  to  item  14) 
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Quick  Reference  Guide  (QRG) 


||  Items  10  through  13  include  only  those  respondents  who  both  indicated  that  they  had  reviewed  the  FLM  QRG  ('Yes'  on 


item  3),  and  had  suggestions  for  improving  the  QRG  ('Yes'  on  item  9). 


n 

10.  Identify  additional  content  areas  that  need  to  be  addressed  in  the  QRG:  24 

1 1 .  Identify  any  potential  content  errors  that  you  found  in  the  QRG:  8 

12.  Identify  any  problems  with  the  organization  or  utility  of  the  QRG:  15 

13.  Do  you  have  any  other  suggestions  or  comments  concerning  the  QRG?  46 

Demographics _ 


Response  Distribution  (%) 


14.  At  which  type  of  facility  are  you  currently  an  active 
Front  Line  Manager?  (Required) 

n 

782 


En  Route  Terminal 

(skip  to  item  17) 


iltems  15  and  16  include  only  those  respondents  who  indicated  that  they  were  an  FLM  at  a  terminal  facility  ('terminal'  on 
litem  14). 


15.  Facility  type: 

n 

439 


16.  Facility  level: 

n 

438 


17.  In  what  year  did  you  become  a  CPC  controller? 

n  min  med  max 

771  1969  1992  2011 


385  35.3 


Control  Tower  with  Tower  Cab  TRACON 

TRACON 


Levels  5-7  Levels  8-10  Levels  11-12 
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Demographics 


1 8.  Were  you  a  CIC?  (Required) 

n 

782 


Response  Distribution  (%) 


88.6 


Yes 


11.4 


(skip  to  item  20) 


Item  19  includes  only  those  respondents  who  indicated  they  were  a  CIC  ('Yes'  on  item  18). 


1 9.  In  what  year  did  you  become  a  CIC? 

n  min  med  max 

661  1973  1995  2010 


20.  In  what  year  did  you  become  a  Front  Line  Manager? 

n  min  med  max 

769  1986  2007  2011 

21.  Have  you  received  a  performance  award? 

n 

773 


22.  Has  your  facility  received  a  performance  award? 

n 

759 


80.9 


Yes  No 


Yes  No 
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